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Abstract 

Many AI researchers are today striving to build agent teams for complex, dynamic 
multi-agent domains, with intended applications in arenas such as education, training, 
entertainment, information integration, and collective robotics. Unfortunately, uncertain- 
ties in these complex, dynamic domains obstruct coherent teamwork. In particular, team 
members often encounter differing, incomplete, and possibly inconsistent views of their en- 
vironment. Furthermore, team members can unexpectedly fail in fulfilling responsibilities 
or discover unexpected opportunities. Highly fiexible coordination and communication is 
key in addressing such uncertainties. Simply fitting individual agents with precomputed 
coordination plans will not do, for their infiexibility can cause severe failures in teamwork, 
and their domain-specificity hinders reusability. 

Our central hypothesis is that the key to such fiexibility and reusability is providing 
agents with general models of teamwork. Agents exploit such models to autonomously rea- 
son about coordination and communication, providing requisite fiexibility. Furthermore, 
the models enable reuse across domains, both saving implementation effort and enforc- 
ing consistency. This article presents one general, implemented model of teamwork, called 
STEAM. The basic building block of teamwork in STEAM is joint intentions (Cohen & 
Levesque, 1991b); teamwork in STEAM is based on agents' building up a (partial) hierar- 
chy of joint intentions (this hierarchy is seen to parallel Grosz & Kraus's partial Shared- 
Plans, 1996). Furthermore, in STEAM, team members monitor the team's and individual 
members' performance, reorganizing the team as necessary. Finally, decision-theoretic com- 
munication selectivity in STEAM ensures reduction in communication overheads of team- 
work, with appropriate sensitivity to the environmental conditions. This article describes 
steam's application in three different complex domains, and presents detailed empirical 
results. 

1. Introduction 

teamwork: cooperative effort by the members of a team to achieve a common 
goal. - American Heritage Dictionary 

Teamwork is becoming increasingly critical in many multi-agent environments, such as, 
virtual training (Tambe et al., 1995; Rao et al., 1993), interactive education (for instance, in 
virtual historical settings, Pimentel & Teixeira, 1994), internet-based information integra- 
tion (Williamson, Sycara, & Decker, 1996), RoboCup robotic and synthetic soccer (Kitano 
et al., 1995, 1997), interactive entertainment (Hayes-Roth, Brownston, & Gen, 1995; Reilly, 
1996), and potential multi-robotic space missions. Teamwork in such complex, dynamic 
domains is more than a simple union of simultaneous coordinated activity. An illustrative 
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example provided by Cohen and Levesque (1991b) — worth repeating, given that the differ- 
ence between simple coordination and teamwork is often unacknowledged in the literature 
— focuses on the distinction between ordinary traffic and driving in a convoy. Ordinary 
traffic is simultaneous and coordinated by traffic signs, but it is not considered teamwork. 
Driving in a convoy, however, is an example of teamwork. The difference in the two situ- 
ations is that while teamwork does involve coordination, in addition, it at least involves a 
common team goal and cooperation among team members. 

This article focuses on the development of a general model of teamwork to enable a 
team to act coherently, overcoming the uncertainties of complex, dynamic environments. 
In particular, in these environments, team members often encounter differing, incomplete 
and possibly inconsistent views of the world and (mental) state of other agents. To act 
coherently, team members must flexibly communicate to avoid miscoordination. Further- 
more, such environments can often cause particular team members to unexpectedly fail 
in fulfilling responsibilities, or to discover unexpected opportunities. Teams must thus be 
capable of monitoring performance, and fiexibly reorganizing and reallocating resources to 
meet any contingencies. Unfortunately, implemented multi-agent systems often fail to pro- 
vide the necessary fiexibility in coordination and communication for coherent teamwork in 
such domains (Jennings, 1994, 1995). In particular, in these systems, agents are supplied 
only with preplanned, domain-specific coordination. When faced with the full brunt of un- 
certainties of complex, dynamic domains, the infiexibility of such preplanned coordination 
leads to drastic failures — it is simply difficult to anticipate and preplan for all possible con- 
tingencies. Furthermore, in scaling up to increasingly complex teamwork situations, these 
coordination failures continually recur. In addition, since coordination plans are domain 
specific, they cannot be reused in other domains. Instead, coordination has to be redesigned 
for each new domain. 

The central hypothesis in this article is that providing agents with a general model 
of teamwork enables them to address such difficulties. Such a model enables agents to 
autonomously reason about coordination and communication, providing them the requisite 
fiexibility in teamwork. Such general models also allow reuse of teamwork capabilities 
across domains. Not only does such reuse save implementation effort, but it also ensures 
consistency in teamwork across applications (Rich & Sidner, 1997). Fortunately, recent 
theories of teamwork have begun to provide the required models for fiexible reasoning about 
teamwork, e.g., joint intentions (Cohen & Levesque, 1991b; Levesque, Cohen, & Nunes, 
1990), SharedPlan (Grosz, 1996; Grosz & Kraus, 1996; Grosz & Sidner, 1990) and joint 
responsibility (Jennings, 1995), are some of the prominent ones among these. However, 
most research efforts have failed to exploit such teamwork theories in building practical 
applications (Jennings, 1994, 1995). 

This article presents an implemented general model of teamwork, called STEAM (sim- 
ply, a Shell for TEAMwork).i At its core, STEAM is based on the joint intentions theory 
(Levesque et al., 1990; Cohen & Levesque, 1991b, 1991a); but it also parallels and in some 
cases borrows from the SharedPlans theory (Grosz, 1996; Grosz & Kraus, 1996; Grosz & 
Sidner, 1990). Thus, while STEAM uses joint intentions as the basic building block of team- 
work, as in the SharedPlan theory, team members build up a complex hierarchical structure 
of joint intentions, individual intentions and beliefs about others' intentions. In STEAM, 

1. STEAM code (with documentation/traces) is available as an online Appendix. 
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communication is driven by commitments embodied in the joint intentions theory — team 
members may communicate to attain mutual belief while building and disbanding joint 
intentions. Thus, joint intentions provide STEAM a principled framework for reasoning 
about communication, providing significant fiexibility. STEAM also facilitates monitoring 
of team performance by exploiting explicit representation of team goals and plans. If in- 
dividuals responsible for particular subtasks fail in fulfilling their responsibilities, or if new 
tasks are discovered without an appropriate assignment of team members to fulfill them, 
team reorganization can occur. Such reorganization, as well as recovery from failures in 
general, is also driven by the team's joint intentions. 

steam's operationalization in complex, real-world domains (described in the next sec- 
tion) has been key in its development to address important teamwork issues discussed above. 
It has also led STEAM to address some practical issues, not addressed in teamwork the- 
ories. One key illustration is in STEAM's detailed attention to communication overheads 
and risks, which can be significant. STEAM integrates decision theoretic communication 
selectivity — agents deliberate upon communication necessities vis-a-vis incoherency in 
teamwork. This decision theoretic framework thus enables improved fiexibility in commu- 
nication in response to unexpected changes in environmental conditions. 

Operationalizing general models of teamwork, such as STEAM, necessitates key modi- 
fications in the underlying agent architectures. Agent architectures such as Soar (Newell, 
1990), RAP (Firby, 1987), PRS (Rao et al., 1993), BBl (Hayes-Roth et al., 1995), and 
IRMA (Pollack, 1992) have so far focused on individual agent's fiexible behaviors via mech- 
anisms such as commitments and reactive plans. Such architectural mechanisms need to be 
enhanced for fiexible teamwork. In particular, an explicit representation of mutual beliefs, 
reactive team plans and team goals is essential. Additional types of commitments, suitable 
for a team context, may need to be embodied in the architectures as well. Without such 
architectural moorings, agents are unable to exploit general models of teamwork, and reason 
about communication and coordination. This view concurs with Grosz (1996), who states 
that "capabilities for teamwork cannot be patched on, but must be designed in from the 
start". 

Our operationalization of STEAM is based on enhancements to the Soar architecture 
(Newell, 1990), plus a set of about 300 domain-independent Soar rules. Three different 
teams have been developed based on this operationalization of STEAM. These teams have 
a complex structure of team-subteam hierarchies, and operate in complex environments 
— in fact, two of them operate in a commercially-developed simulation environment for 
training. This article presents detailed experimental results from these teams, illustrating 
the benefits of STEAM in their development. 

STEAM is among just a very few implemented general models of teamwork. Other 
models include Jennings' joint responsibility framework in the GRATE* system (Jennings, 
1995) (based on Joint Intentions theory), and Rich and Sidner's COLLAGEN (Rich & Sid- 
ner, 1997) (based on the SharedPlans theory), that both operate in complex domains. While 
Section 7 will discuss these in greater detail, STEAM significantly differs from both these 
frameworks, via its focus on a different (and arguably wider) set of teamwork capabilities 
that arise in domains with teams of more than two-three agents, with more complex team 
organizational hierarchies, and with practical emphasis on communication costs. 
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The rest of the article begins with a concrete motivation for our research via a description 
of key teamwork problems in real-world domains (Section 2). Section 3 discusses theories of 
teamwork and sketches their implications for STEAM. Section 4 next describes STEAM, our 
implemented model of teamwork. Section 5 discusses STEAM's selective communication. 
Section 6 presents a detailed experimental evaluation. Section 7 discusses related work. 
Finally, Section 8 presents summary and future work. 



2. Illustrative Domains and Motivations 

This investigation focuses on three separate domains. Two of the domains are based on a 
real-world distributed, interactive simulator commercially developed for military training 
(Calder et al., 1993). The simulator enables — via networking of several computers — 
creation of large-scale, 3D synthetic battlefields, where humans, as well as hundreds or even 
thousands of intelligent and semi-intelligent agents can co-participate (Tambe et al., 1995). 

The first domain. Attack (Figure 1), involves pilot agents for a company of (up to eight) 
synthetic attack helicopters. The company starts at the home-base, where the commander 
pilot agent first sends orders and instructions to the company members. The company 
processes these orders and then begins fiying towards their specified battle position, i.e., 
the area from which the company will attack the enemy. While enroute to the battle- 
position, depending on the orders, the company members may fiy together or dynamically 
split into pre-determined subteams. Once the company reaches a holding point, it halts. 
One or two scout helicopters fiy forward and first scout the battle position. Based on 
communication from the scouts, other company members fiy forward to the battle position. 
Here, individual pilots repeatedly mask(hide) their helicopters and unmask to shoot missiles 
at enemy targets. Once the attack completes, the helicopters regroup and return to their 
home-base. While enroute to the home-base (or initially towards the battle-position), if 
any company member spots enemy vehicles posing a threat to the company, it alerts others. 
The company then evades and bypasses the enemy vehicles, while also protecting itself 
using guns. When the company returns safely to home-base, it rearms and refuels, readying 
itself for the next mission. An overview of the overall research and development effort in 
this domain, simulation infrastructure, milestones, and agent behaviors is presented in (Hill 
et al., 1997). 



HOLDING BRIDGE 
POINT. 



ENEMY 

positionV/ vehicles 



Figure 1: Attack domain: company fiying in subteams 

In the second domain. Transport (Figure 2), synthetic transport helicopters protected 
by escort helicopters fiy synthetic troops to land. In a typical mission, two or four escort 
helicopters and four to twelve transport helicopters take off from separate ships at sea to 
rendezvous at a link-up point. The escorts then provide a protective cover to the transport 
helicopters during the entire fiight to and from their pre-specified landing zone (where the 
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synthetic troops dismount). This domain may involve teams of up to sixteen synthetic pilot 
agents (the largest team we have encountered); although Figure 2 shows twelve. 




Figure 2: Transport domain with synthetic escort and transport helicopters. 

Our third domain is RoboCup synthetic soccer (Kitano et al., 1995). RoboCup is 
an international soccer tournament for robots and synthetic agents, aimed at promoting 
research in multi-agent systems. In the synthetic agent track, over 30 teams will participate 
in the first RoboCup'97 tournament at IJCAF97 in Japan. The snapshot in Figure 3 shows 
two competing teams: CMUnited (Stone & Veloso, 1996) versus our 75*/ team. ^ 

The Attack domain is illustrative of the teamwork challenges. In our initial, pre-STEAM 
implementation, the helicopter pilot agents were developed in the Soar integrated agent- 
architecture (Newell, 1990; Rosenbloom et al., 1991). Each pilot agent was based on a 
separate copy of Soar. For each such pilot, an operator hierarchy was defined. Figure 
4 shows a portion of this hierarchy (Tambe, Schwamb, & Rosenbloom, 1995). Operators 
are very similar to reactive plans commonly used in other agent architectures, such as 
the architectures described in Section 1. Each operator consists of (i) precondition rules 
for selection; (ii) rules for application (a complex operator subgoals); and (iii) rules for 
termination. At any one point, only one path through this hierarchy is active, i.e., it governs 
an individual's behavior. For teamwork among individuals, domain-specific coordination 
plans were added, as commonly done in other such efforts in this type of domain (Rajput & 
Karr, 1995; Tidhar, Selvestrel, & Heinze, 1995; Laird, Jones, & Nielsen, 1994; Coradeschi, 
1997), including our own (Tambe et al., 1995). For instance, after scouting the battle 
position, a scout executes a plan to inform those waiting behind that the battle position is 
scouted (not shown in Figure 4). 

Initially, with two-three pilot agents and few enemy vehicles, limited behaviors and con- 
trolled agent interaction, carefully preplanned coordination was adequate to demonstrate 
desired behaviors. However, as the numbers of agents and vehicles increased, their behav- 
iors were enriched, and domain experts (human pilots) began to specify complex missions, 
significant numbers of unanticipated agent interactions surfaced. Faced with the full brunt 

2. Since March 1997, a team of graduate students at the Information Sciences Institute (ISI) has joined 
in in further research and maintenance of the ISI team. While the author continues to be responsible 
for the teamwork in the player agents, others have made significant contributions to individual agent 
behaviors. 
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Figure 3: The Robocup synthetic soccer domain. 
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location 

Figure 4: Attack domain: Portion of a pilot agent's operator hierarchy. 

of the uncertainties in this complex, dynamic environment, the carefully hand-coded, pre- 
planned coordination led to a variety of teamwork failures in the various demonstrations 
and exercises in 1995-96. Figure 5 lists a small sample of the teamwork failures, roughly in 
the order they were encountered. 

One approach to address these failures is a further addition of domain-specific coordi- 
nation plans; and indeed, this was the first approach we attempted. However, there are 
several difficulties. First, there is no overarching framework that would enable anticipation 
of teamwork failures; the teamwork failures just appear to arise unexpectedly. As a result. 
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1. Upon abnormally terminating engagement with the enemy, the company commander returned 
to home base alone, abandoning members of its own company at the battle position. 

2. Upon reaching the holding area, the company waited, while a single scout started flying 
forward. Unfortunately, the scout unexpectedly crashed into a hillside; now, the rest of the 
company just waited indeflnitely for the scout's scouting message. 

3. One pilot agent unexpectedly processed its initial orders before others. It then flew towards 
the battle position, while its teammates were left behind at the home base. 

4. Only a scout made it to the holding area (all other helicopters crashed or got shot down); 
but the scout scouted the battle position anyhow, and waited indeflnitely for its non-existent 
company to move forward. 

5. When the initial orders unexpectedly failed to allocate the scouting role to team members, 
the company members waited indeflnitely when they reached the holding point. 

6. Instructions sent by the commander pilot agent to some company members were lost, because 
the commander unexpectedly sent them while the members were busy with other tasks. Hence, 
these members were unable to select appropriate actions. 

7. While evading an enemy vehicle encountered enroute, one helicopter pilot agent unexpectedly 
destroyed the vehicle via gunflre. However, this pilot agent did not inform others; and thus 
an unnecessary, circuitous bypass route was planned. 

8. In an extreme case, when all company members ran out of ammunition, the company failed 
to infer that their mission could not continue. 

9. Two separate companies of helicopters were accidentally allowed to use the same radio chan- 
nels, leading to interference and loss of an initial message from one of the company commanders 
— its company hung indeflnitely. 

Figure 5: Some illustrative examples of breakdown in teamwork. 

coordination plans have to be added on a case-by-case basis — a difficult process, since fail- 
ures have to be first encountered in actual runs. Furthermore, as the system continues to 
scale up to increasingly complex teamwork scenarios, such failures continually recur. Thus, 
a large number of special case coordination plans are potentially necessary. Finally, it is 
difficult to reuse such plans in other domains. 

Given these difficulties, we have pursued an alternative approach — provide agents 
with a general model of teamwork. The agents can then themselves reason about their co- 
ordination/communication responsibilities as well as anticipate and avoid (or recover from) 
teamwork failures. Such an approach also requires an explicit representation of agents' team 
goals and team plans; for that is the very basis for reasoning about teamwork. Unfortu- 
nately, the agent's operator hierarchy shown in Figure 4 represents its own activities. Thus, 
although the agent is provided information about its teammates, their participation in par- 
ticular activities is not explicit (but rather, implicit in the coordination plans). As a result, 
the agent remains ignorant as to which operators truly involve teamwork and the teammates 
involved in them. For instance, execute-mission and engage are in reality team activities 
involving the entire company; while mask and unmask involve no teamwork. Furthermore, 
in some team tasks only subteams are involved, adding to the difficulty of relying on implicit 
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representations since the teammates involved in team tasks vary. Even more problematic 
for implicit representation are team tasks where the team members perform non-identical 
activities. For instance, consider team tasks such as travelling overmatch (where one sub- 
team travels while the other overwatches), or wait while battle position scouted (where scouts 
scout the battle position while the non-scouts wait). In such tasks, no single agent performs 
the team activity, and yet it is important to represent and reason about the combined ac- 
tivity that results. This difficulty in representation is not specific to the Soar architecture, 
but the entire family of architectures mentioned in Section I. 

More importantly, concomitant with the explicit team goals and plans are certain com- 
mitments and coordination responsibilities towards the team, based on the general model 
of teamwork employed. In the absence of both the explicit representation of team goals and 
plans, as well as commitments and responsibilities they engender, agents are often forced 
to rely on the problematic domain-specific coordination plans, leading to aforementioned 
teamwork failures. 

3. Models of Teamwork 

Several teamwork theories have been proposed in the literature (Cohen & Levesque, 1991b; 
Grosz & Kraus, 1996; Jennings, 1995; Kinny et al., 1992). The theories are not intended 
to be directly implemented (say via a theorem prover), but to be used as a specification 
for agent design. They often prescribe general, rather than domain-specific, reasoning 
processes or heuristics for teamwork. Different types of operational teamwork models could 
potentially emerge from these theories — the space of such models remains to be fully 
explored and understood. In developing STEAM, we have focused on the joint intentions 
theory (Cohen & Levesque, 1991b; Levesque et al., 1990; Cohen & Levesque, 1991a), given 
its detailed formal specification and prescriptive power. The joint intentions theory is 
briefiy reviewed in Section 3.1. STEAM ultimately builds on joint intentions in a way that 
parallels the SharedPlan theory (Grosz & Sidner, 1990; Grosz, 1996; Grosz & Kraus, 1996). 
The SharedPlans theory is very briefiy reviewed in Section 3.2. Section 3.3 sketches the 
implications of the theories for STEAM. It outlines the rationale for the design decisions 
in STEAM — in the process, it briefiy compares the capabilities provided by the joint 
intentions and SharedPlan theories. STEAM is later presented in detail in Sections 4 and 
5. 

3.1 Joint Intentions Theory 

The joint intentions framework (Cohen & Levesque, 1991b, 1991a; Levesque et al., 1990) 
focuses on a team's joint mental state, called a joint intention. A team jointly intends a 
team action if team members are jointly committed to completing that team action, while 
mutually believing that they were doing it. A joint commitment in turn is defined as a joint 
persistent goal (JPG). The team 0's JPG to achieve p, where p stands for completion of a 
team action, is denoted JPG(0, p, q). q is an irrelevance clause — as described below, it 
enables a team to drop the JPG should they mutually believe q to be false. JPG(0, p, q) 
holds iff three conditions are satisfied: 

1. All team members mutually believe that p is currently false. 
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2. All team members have p as their mutual goal, i.e, they mutually know that they want p to 
be eventually true. 

3. All team members mutually believe that until p is mutually known to be achieved, unachiev- 
able or irrelevant they mutually believe that they each hold p as a weak goal (WAG).^ WAG(//, 
p, 0, q), where is a team member in 0, implies that one of the following holds: 

• /J, believes p is currently false and wants it to be eventually true, i.e., p is a normal 
achievement goal); or 

• Having privately discovered p to be achieved, unachievable or irrelevant (because q is 
false), /J, has committed to having this private belief become 0's mutual belief. 

JPG provides a basic change in plan expressiveness, since it builds on a team task p. 
Furthermore, a JPG guarantees that team members cannot decommit until p is mutually 
believed to be achieved, unachievable or irrelevant. Basically, JPG(0, p, q) requires team 
members to each hold p as a weak achievement goal (WAG). WAG(p, p, 0, q), where is 
a team member in 0, requires to adopt p as its goal if it believes p to be false. However, 
should 1^ privately believe that p has terminated — i.e., p is either achieved, unachievable 
or irrelevant — JPG(0,p, q) is dissolved, but is left with a commitment to have this 
belief become 0's mutual belief. To establish mutual belief, must typically communicate 
with its teammates about the status of the team task p. 

The commitment to attain mutual belief in the termination of p is a key aspect of a 
JPG. This commitment ensures that team members stay updated about the status of team 
activities, and thus do not unnecessarily face risks or waste their time. For instance, consider 
the first failure presented in Section 5, where the commander returned to home base alone, 
abandoning its teammates to face a risky situation. Such failures can be avoided given the 
commitments in a JPG. In our example, the commander would have communicated with 
its teammates to establish mutual belief about the termination of the engagement. 

To enter into a joint commitment (JPG) in the first place, all team members must 
establish appropriate mutual beliefs and commitments. An explicit exchange of request 
and confirm speech acts is one way that a team can achieve appropriate mutual beliefs 
and commitments (Smith & Cohen, 1996). Since this exchange leads to establishment of 
a JPG, we will refer to it in the following as the establish commitments protocol. The key 
to this protocol is a persistent weak achievement goal (PWAG). PWAG(//i, p, 0) denotes 
commitment of a team member z/i to its team task p prior to the team's establishing a 
JPG."* 1^ initiates the protocol while its teammates in 0, z/l,.,z/i..z/n, respond: 

1. /J, executes a Request(//, 0, p), cast as an Attempt(//, (f), ip). That is, //'s ultimate goal (f) is 
to both achieve p, and have all i>i adopt PWAG(i^i, p, 0). However, /j, is minimally committed 
to ip, where ip denotes achieving mutual belief in that /j, has the PWAG to achieve (p. With 
this Request, /j, adopts the PWAG. 

2. Each i>i responds via confirm or refuse. Confirm, also an Attempt, informs others that 
i>i has the PWAG to achieve p. 

3. WAG was originally called WG in (Levesque et al., 1990), but later termed WAG in (Smith & Cohen, 
1996). 

4. The PWAG also includes an irrelevance clause q, but we will not include it here to simplify the following 
description. 
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3. If V i, m confirm, JPG(0, p) is formed. 

In establishing a JPG, this protocol synchronizes 0. In particular, with this protocol, 
members simultaneously enter into a joint commitment towards a current team activity 
p. While the JPG is the end product of the establish commitment protocol, important 
behavioral constraints are enforced during execution via the PWAGs. In step 1, the adop- 
tion of a PWAG implies that if after requesting, privately believes that p is achieved, 
unachievable or irrelevant, it must inform its teammates. Furthermore, if believes that 
the minimal commitment tp is not achieved, it must retry (e.g., if a message did not get 
through it must retransmit the message). Step 2 similarly constrains team members z/i to 
inform others about p, and to rebroadcast. As step 3 indicates, all team members must 
consent, via confirmation, to the establishment of a JPG. A JPG is not established if any 
one agent refuses. Negotiations among team members may ensue in such a case; however, 
that remains an open issue for future work. 

3.2 Shared Plans Theory 

In contrast with joint intentions, the concept of SharedPlans (SP) is not based on a joint 
mental attitude (Grosz, 1996; Grosz & Kraus, 1996; Grosz & Sidner, 1990). Instead, SP 
relies on a novel intentional attitude, intending that, which is similar to an agent's normal 
intention to do an action. However, an individual agent's intention that is directed towards 
its collaborator's actions or towards a group's joint action. Intention that is defined via a 
set of axioms that guide an individual to take actions, including communicative actions, 
that enable or facilitate its teammates, subteam or team to perform assigned tasks (Grosz 
& Kraus, 1996). 

An SP is either a full SharedPlan (FSP) or a partial SharedPlan(FSF ) . We will begin 
with a definition of an FSP, and then follow with brief remarks about a PSP. An FSP to 
do a represents a situation where every aspect of a joint activity a is fully determined. 
This includes mutual belief and agreement in the complete recipe R„ to do a. R„ is 
a specification of a set of actions (3i, which when executed under specified constraints, 
constitutes performance of a. FSP(P, GR, a, Tp, T„, R„) denotes a group Grid's plan P 
at time Tp to do action a at time T„ using recipe R„. Very briefiy, FSP(P, GR, a, Tp, 
T„, R„) holds iff the following conditions are satisfied:^ 

1. All members of group GR mutually believe that they each intend that the proposition Do(GR, 
a, Ta) holds i.e., that GR does a over time T„. 

2. All members of GR mutually believe that R„ is the recipe for a. 

3. For each step l3i in R„: 

• A subgroup GRk (GRk C GR) has an FSP for l3i, using recipe R^j-. (GRk may only be 
an individual, in which case, it must have a full individual plan, an analogue of FSP for 
individuals.) 

• Other members of GR believe that there exists a recipe such that GRk can bring about 
l3i and have an FSP for l3i (but other members may not know R^j). 

5. For the sake of brevity, a context clause Cc, is deleted from this definition. Also, in this article, we will 
not address the contracting csLse discussed in (Grosz & Kraus, 1996). 
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• Other members of GR intend that GRk can bring about l3i using some recipe. 

The SharedPlan theory aspires to describe the entire web of a team's intentions and 
beliefs when engaged in teamwork. In this endeavor, an FSP represents a limiting case; 
usually, when engaged in a team activity, a team only has a partial SharedPlan (PSP). The 
PSP is a snapshot of the team's mental state in a particular situation in their teamwork, 
and further communication and planning is often used to fulfill the conditions of an FSP 
(although, in dynamic domains, the team may never actually form an FSP). We focus on 
three relevant arenas in which partiality may exist in a PSP. First, the recipe R„ may be 
only partially specified. Certainly, in dynamic environments, such as the ones of interest 
in our work, recipes could be considered to evolve over time, as teams reactively decide 
the next step based both on the context and the current situation. For instance, in the 
Attack domain, the helicopter company may react to enemy vehicles seen enroute, thus 
evolving their recipe. According to SP theory, team member must arrive at mutual belief 
in their next step(s) (3i. For each step (3i in the recipe, the relevant subgroup must form a 
SharedPlan. 

Second, the team's task allocation may be unreconciled, e.g., the agent or group to 
perform particular task may not be determined. In this situation, team members intend 
that there exist some individual or subgroup to do the task. Among actions considered as 
a result of the intending that, individuals may volunteer to perform the unreconciled task, 
or persuade/order others to take over the task. 

Third, individuals or subgroups may not have attained appropriate mutual beliefs for 
forming an FSP, leading to communication within the team. Communication may also 
arise due to agents' "intention that" attitude both towards their team goal and towards 
teammates' activities. For instance, a team member's intention that its team do an action (3i, 
and its belief that communication of some particular information will enable the team to do 
(3i, will lead it to communicate that information to the team (as long as such communication 
does not conflict with previous commitments). 

3.3 The Influence of Teamwork Theories on STEAM 

In STEAM, joint intentions are used as building blocks of teamwork. Several advantages 
accrue due to this use. First, the commitments in a joint intention begin to provide a 
principled framework for reasoning about coordination and communication in teamwork. 
Thus, this framework begins to address teamwork failures such as those in Figure 5. Second, 
the joint commitments in joint intentions provide guidance for monitoring and maintenance 
of a team activity, i.e., agents should monitor conditions that cause the team activity to 
be achieved or unachievable or irrelevant, and maintain the team activity at least until one 
of these conditions arises. Third, a joint intention leads to an explicit representation of a 
team activity, and thus facilitates reasoning about teamwork. In particular, as shown later, 
agents can reason about the relationship between their team activity and an individual's or 
subteam's contributions to it. 

However, a single joint intention for a high-level team goal a is insufficient to provide 
all of these advantages. To guarantee coherent teamwork, four additional issues must be 
addressed. Here, the SharedPlans theory helps in analysis of STEAM's approach, and in 
one case, STEAM directly borrows from SharedPlans. A key observation is that analogous 
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to partial SharedPlans, STEAM builds up snapshots of the team's mental state, but via 
joint intentions. 

The first issue involves coherence in teamwork — team members must pursue a common 
solution path in service of their joint intention for the high-level team goal a. Indeed, as 
Jennings (1995) observes, without such a constraint, team members could pursue alternative 
solution paths that cancel each other, so no progress is made towards a. The SharedPlan 
theory addresses such coherence by stepping beyond the team members' "intentions that" 
towards a. In addition, SharedPlans mandates mutual belief in a common recipe (even if 
partial) and SharedPlans for individual steps (3i in the common recipe, thus generating a 
recursive hierarchy to ensure coherence. 

steam's approach here parallels that of SharedPlans; however, it builds on joint inten- 
tions rather than SharedPlans. That is, STEAM uses joint intentions as a building block 
to hierarchically build up the mental attitude of individual team members, and ensure that 
team members pursue a common solution path. In particular, as mentioned earlier, in 
dynamic domains, given reactive plans, a recipe R„ may evolve step by step during execu- 
tion. In STEAM, as the recipe evolves, if a step (3i requires execution by the entire team, 
STEAM requires that the entire team agree on (3i, and form joint intentions to execute it. 
To execute a substep of (3i, other joint intentions are formed, leading to a hierarchy. During 
the expansion of this hierarchy, if a step involves only a subteam then that subteam must 
form a joint intention to perform that step. If only an individual is involved in the step, it 
must form an intention to do that step. In general, the resulting intention hierarchy evolves 
dynamically, depending on the situations the team encounters. 

Second, Grosz and Kraus (1996) discuss the tradeoffs in the amount of information team 
members must maintain about teammates' activities, particularly when a step (3i involves 
only a subteam, or an individual. Grosz and Kraus address this tradeoff in SharedPlans 
as shown in step 3b in Section 3.2, requiring that team members know only that a recipe 
exists to enable a teammate(s) to perform its actions, but not the details of the recipe. 
Similarly, STEAM requires that in case a step (3i is performed by a subteam (or just an 
individual team member), remaining team members track the subteam's joint intention (or 
the relevant team member's intention) to perform the step. This intention tracking need 
not involve detailed plan recognition, e.g., as in our previous work (Tambe, 1995, 1996). 
Instead, a team member must only be able to infer that its teammates intend (or cannot 
or do not intend) to execute the step (3i. This minimal constraint is necessary because 
otherwise, team members may be unable to monitor the current status of the team activity, 
e.g., that their team activity has fallen apart. In addition, some information about the 
dependency relationship among team members' actions is useful in monitoring, as discussed 
in Section 4.2. 

A third issue is the analogue of the "unreconciled" case in SharedPlans. STEAM forms 
a joint intention to replan whenever a team's joint intention for a step (3i is seen to be 
unachievable. Replanning may lead the team to first analyze the cause of the initial un- 
achievability. Among other possibilities, the cause could be the absence of assignment of 
a subtask to a subteam or individual, or the failure of the relevant individual or subteam 
in performing the subtask. In such a case, each team member acts to determine the ap- 
propriate agent or subteam for performing the relevant task. As a result, an agent can 
volunteer itself, or suggest to other individuals or subteams to perform the unassigned task. 



94 



Towards Flexible Teamwork 



Of course, the unachievability may be the result of other causes besides lack of assignment; 
replanning must then address this other cause (further discussion in Section 4.2). 

A final issue is generalization of STEAM's communication capabilities via a hybrid ap- 
proach that combines the prescriptions of the joint intentions approach with some aspects 
of SharedPlans. A key observation based on (Grosz & Kraus, 1996) is that the communica- 
tion in joint intentions could potentially be arrived at in SharedPlans via axioms defining 
intention that. For instance, consider that a team member has obtained private informa- 
tion about the achievement of the team's current team action (3i. In joint intentions, this 
team member will seek to attain mutual belief in the achievement of /3i, leading to com- 
munication. In contrast, in SharedPlans, the team member's communication would arise 
because: (i) it intends that the team do some action (32 which follows /3i, and (ii) the team 
cannot do (32 without all team members being aware of achievement of (3i. Thus, further 
first principles reasoning, based on interrelationships among actions, is required to derive 
relevant communication in SharedPlans; but in this instance, joint intentions provide for 
such communication without the reasoning. 

In general, if the team's termination of one action (3i is essential for the team to perform 
some following action (32, the prescription in joint intentions — to attain mutual belief in 
termination of team actions — is adequate for relevant communication. However, in some 
cases, additional communication based on specific information-dependency relationships 
among actions is also essential. For instance, the scouts in the Attack domain not only 
inform all company members of completion of their scouting activity (so the company can 
move forward), but also the precise coordinates of enemy location to enable the company 
to occupy good attacking positions (information-dependency). Such communication could 
also be potentially derived from the axioms of intention that in SharedPlans, but at the 
cost of further reasoning. 

STEAM does not rely on the first-principles reasoning from intention that for its commu- 
nication, relying on the prescriptions of joint intentions instead. However, STEAM exploits 
explicit declaration of information-dependency relationships among actions, for additional 
communication. Thus, when communicating the termination of a team action (3i, STEAM 
checks for any inferred or declared information-dependency relationships with any following 
action (3j. The information relevant for (3j is also communicated when attaining mutual 
belief in the termination of (3i. As a result, based on the specific information-dependency 
relationship specified, different types of information are communicated, when terminating 
(3i. Thus, the scouts can communicate the location of enemy units when communicating 
the completion of their scouting - given the information-dependency relationship with the 
planning of attacking positions. If no such relationship is specified, or if other relationships 
are specified, the scouts would communicate different information. 

STEAM thus starts with joint intentions, but then builds up hierarchical structures 
that parallel the SharedPlans theory, particularly, partial SharedPlans. The result could 
be considered a hybrid model of teamwork, that borrows from the strengths of both joint 
intentions (formalization of commitments in building and maintaining joint intentions) and 
SharedPlans (detailed treatment of team's attitudes in complex tasks, as well as unrecon- 
ciled tasks). This is of course not the only possible hybrid. As mentioned earlier, further 
exploration in the space of teamwork models is clearly essential. 
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4. STEAM 

steam's basis is in executing hierarchical reactive plans, in common with architectures 
mentioned in Section 1. The novel aspects of STEAM relate to its teamwork capabilities. 
The key novelty in STEAM is team operators (reactive team plans). When agents developed 
in STEAM select a team operator for execution, they instantiate a team's joint intentions. 
Team operators explicitly express a team's joint activities, unlike the regular "individual 
operators" which express an agent's own activities. In the hierarchy in Figure 6, operators 
shown in [] such as [Engage] are team operators, while others are individual operators. 
Team activities such as travelling overmatch or waiting while battle position scouted are now 
easily expressed as team operators, as shown in Figure 6, with activities of individuals or 
subteams expressed as children of these operators. (Team operators marked with "*" are 
typically executed by subteams in this domain.) 
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Figure 6: Attack domain: Portion of modified operator hierarchy with team operators. 

As with individual operators, team operators also consist of: (i) precondition rules; (ii) 
application rules; and (iii) termination rules. Whether an operator is a team operator or 
an individual operator is dynamically determined. In particular, when an agent v\ invokes 
an operator for execution, the operator is annotated with an "executing agent", which may 
be dynamically determined to be an individual, or subteam, or a team. If the "executing 
agent" is a particular team or subteam, the operator is determined to be a team operator. 
If the "executing agent" is the agent v\ itself, then an individual operator results. Thus, 
precise team executing a team operator is not compiled in, but can be fiexibly determined 
at execution time. Figure 6 thus illustrates the configuration of operators that is typical in 
the Attack domain. 

Given an arbitrary team operator OP, all team members must simultaneously select OP 
to establish a joint intention (joint intention for OP will be denoted as [OP]e)- In Figure 
6, at the highest level, the team forms a joint intention for [execute-mission] e . In service of 
this joint intention, the team may form a joint intention [engage]e. In service of [engage]e, 
individual team members all select individual operators to employ- weapons, thus forming 
individual intentions. An entire hierarchy of joint and individual intentions is thus formed 
when an agent participates in teamwork. 
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A STEAM-based agent maintains its own private state for the application of its indi- 
vidual operators; and a "team state" to apply team operators. A team state is the agent's 
(abstract) model of the team's mutual beliefs about the world, e.g., in the Transport do- 
main, the team state includes the coordinates of the landing zone. The team state is usually 
initialized with information about the team, such as the team members in the team, possible 
subteams, available communication channels for the team, the pre-determined team leader 
and so forth. STEAM can also maintain subteam states for subteam participation. There 
is of course no shared memory, and thus each team member maintains its own copy of the 
team state, and any subteam states for subteams it participates in. To preserve the con- 
sistency of a (sub)team state, one key restriction is imposed for modifications to it — only 
the team operators representing that (sub)team's joint intentions can modify it. Thus, the 
state corresponding to a subteam i7 can only be modified in the context of a joint intention 
[OP]q. 

Thus, at minimum, STEAM requires the following modifications to the architectures 
such as Soar, RAP, PRS and others mentioned in Section 1 to support teamwork: (i) gen- 
eralization of operators (reactive plans) to represent team operators (reactive team plans); 
(ii) representation of team and/or subteam states, and (iii) restrictions on team state mod- 
ifications (only via appropriate team operators). While these team operators and team 
states are at the foundation of STEAM, as a general model of teamwork, STEAM also 
involves agents' commitments in teamwork, monitoring and replanning capabilities, and 
more. Hard-wiring this entire teamwork model within the agent architectures could po- 
tentially lead to unnecessary rigidity in agent behaviors. Instead, the STEAM model is 
maintained as a domain-independent, operational module (e.g., in the form of rules) to 
guide agents' behaviors in teamwork. In the future, appropriate generalizations of these 
capabilities could begin to be integrated in agent architectures. 

The following subsections now discuss key aspects of STEAM in detail. Section 4.1 
discusses team operator execution in STEAM. Section 4.2 describes STEAM's capabilities 
for monitoring and replanning. Detailed pseudo-code for executing STEAM appears in 
Appendix A. 

4.1 Team Operator Execution 

To execute a team operator, agents must first establish it as a joint intention. Thus, when 
a member selects a team operator for execution, it first executes the establish commitments 
protocol described below (introduced in Section 3.1): 

1. Team leader broadcasts a message to the team to establish PWAG to operator OP. Leader 
now establishes PWAG. If [OP]© not established within time limit, repeat broadcast. 

2. Subordinates i>i in the team wait until they receive leader's message. Then, turn by turn, 
broadcast to establishment of PWAG for OP; and establish PWAG. 

3. Wait until V m, vi establish PWAG for OP; establish [OP]©. 

With this establish commitment protocol, agents avoid problems of the type where just 
one member files off to the battle position (item 3, Figure 5). In particular, a team member 
cannot begin executing the mission without first establishing a joint intention [execute- 
mission]e. During execution of the establish commitment protocol, PWAGs address several 
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contingencies — if an OP is believed achieved, unachievable or irrelevant prior to [OP]e, 
agents inform teammates. Other contingencies are also addressed, e.g., even if a subordinate 
initially disagrees with the leader, it will conform to the leader's broadcasted choice of 
operators. In general, resolving disagreements among team members via negotiation is a 
significant research problem in its own right (Chu-CarroU & Carberry, 1996), which is not 
addressed in STEAM. Instead, currently STEAM relies on a team leader to initiate the 
request, and thus resolve disagreements. 

After establishing a joint intention [OP]e, a team operator can only be terminated by 
updating the team state (mutual beliefs). This restriction on team operator termination 
avoids critical communication failures of the type where the commander returned to home- 
base alone — instead, agents must now inform teammates when terminating team operators. 
Furthermore, with each team operator, multiple termination conditions may be specified, 
i.e., conditions that make the operator achieved, unachievable or irrelevant. Now, if an 
agent's private state contains a belief that matches with a team operator's termination 
condition, and such a belief is absent in its team state, then it creates a communicative 
goal, i.e., a communication operator. This operator broadcasts the belief to the team, 
updating the team state, and then terminating the team operator. 

As mentioned earlier, during teamwork, an agent may be a participant in several joint 
intentions, some involving the entire team, some only a subteam. Thus, an agent may be 
participating in a joint intention involving the entire company, such as [execute-mission] e, 
as well as one involving just a subteam, such as [mask-and-observe]Q. When the termina- 
tion condition of a specific team operator is satisfied, a STEAM-based agent will aim to 
attain mutual belief in only the relevant subteam or team, e.g., facts relevant to [mask-and- 
observejo may only be communicated among i7. 

During the broadcast of the communication message, STEAM checks for information- 
dependency relationships with any following tasks; if one exists, relevant information is 
extracted from the current world state and broadcast as well. The information-dependency 
relationship may be specified individually per specific termination condition. For instance, 
suppose a company member z/4 sees some enemy tanks on the route while fiying to home 
base. It recognizes that this fact causes the team's current joint intention [fiy-fiight-plan]e 
to be unachievable. If this fact is absent in the team state, then a communication operator 
is executed, resulting in a message broadcast indicating termination of the fly-flight-plan 
team operator. In addition, STEAM uses the explicitly specified information-dependency 
relationship with a following operator evade to extract the x,y location and direction of the 
tank. As a result, the following communication is generated: 

z/^ terminate-JPG fly-flight-plan evade tank elaborations 61000 4IOOO right. 

This message identifies the speaker (/^4), and informs team members to terminate [fiy- 
fiight-plan]e in order to evade a tank. Thus, z/4 informs others; it does not evade tanks 
on its own. The part of z/4's message that follows the key word elaborations is due to the 
information-dependency relationship. This information — the x,y location and direction of 
the tank — enables team members to evade appropriately. Separating out the information- 
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dependency component in this fashion provides additional communication flexibility, as 
explained earlier in Section 3.3.^ 

4.2 Monitoring and Replanning 

One major source of teamwork failures, as outlined in Section 2, is agents' inability to 
monitor team performance. STEAM facilitates such monitoring by exploiting its explicit 
representation of team operators. In particular, STEAM allows an explicit speciflcation of 
monitoring conditions to determine achievement, unachievability or irrelevancy of team op- 
erators. In addition, STEAM facilitates explicit speciflcation of the relationship between a 
team operator and individuals' or subteam's contributions to it. STEAM uses these specifl- 
cations to infer the achievement or unachievability of a team operator. These speciflcations 
are based on the notion of a role. A role is an abstract speciflcation of the set of activities 
an individual or a subteam undertakes in service of the team's overall activity. Thus, a role 
constrains a team member z/i (or a subteam i7) to some suboperator(s) opj^j- of the team 
operator [OP]e- For instance, suppose a subteam i7 is assigned the role of a scout in the 
Attack domain. This role constrains the subteam i7 to execute the suboperator(s) to scout 
the battle position in service of the overall team operator wait-while-hattle-position-scouted 
(see Figure 6). 

Based on the notion of roles, three primitive role-relationships (i) AND-combination 
(ii) OR-combination and (iii) Role-dependency can currently be specifled in STEAM. These 
primitive role-relationships — called role-monitoring constraints — imply the following 
relationships between a team operator [OP] and its suboperators: 

1. AND-combmation: [OP]© <;=^ /\"^^ op^i 

2. OR- comb matt on: [OP]© <S=^ Vfci opi^i 

3. Role dependency: op^j- op^j (op^j- dependent on op^j) 

These primitive role-monitoring constraints may be combined, to specify more com- 
plex relationships. For instance, for three agents z/i, z/j and z/k, with roles opj^i, opj,j and 
opj,fc, a combination AND-OR role relationship can be specifled as ((opj^j- V opj,j) /\ opj^^). 
STEAM-based agents can now infer that the role non-performance of z/k (-■opj.fc) makes 
OPe unachievable; but the role non-performance of just one of z/i or z/j is not critical to 
OPe- Similarly, for two agents z/i and z/j, both an OR-combination plus role-dependency 
may be specifled as ((opj^j- V opj,j) /\ (opj^j- =^ opj,j)). Role monitoring constraints may 
be specifled in terms of individuals' roles, or subteam's roles. 

The mechanisms for tracking teammates' role performance or inferring their role non- 
performance is partly domain dependent. As mentioned in Section 3.3, in some domains, 
an agent need not know its teammate's detailed plan or track that in detail, but may rely 
on high-level observations. For instance, in the Attack domain, if a helicopter is destroyed, 
team members infer role non-performance for the affected team member. In other cases, 
such as the RoboCup Soccer domain, no such high-level indication is available. Instead, 

6. In the future, to enable STEAM-based agents to communicate with non-STEAM-based agents, a generic 
communication language may be necessary. While generating natural language is currently outside the 
scope of STEAM, STEAM does not preclude such a possibility. Alternatively, an artificial communication 
language, such as (Sidner, 1994) may be used. 



99 



Tambe 



agents need to obtain role performance information via agent tracking (plan recognition) 
(Tambe, 1995, 1996), e.g., is a player agent in the RoboCup simulation dashing ahead to 
receive a pass? Communication may be another source of information regarding role non- 
performance. First, as discussed below, STEAM leads individuals to announce role-changes 
to the team, and thus other team members indirectly infer role-performance information. 
Second, as discussed in Section 5.1, STEAM may lead individuals to directly communicate 
their role non-performance. Additionally, a few domain-independent mechanisms for infer- 
ring role performance are provided in STEAM. Thus, role non-performance is inferred if no 
individual or subteam is specified for performance of a role (as in item 5, Figure 5). Also, 
if all individuals within a subteam are found incapable of performing their roles, STEAM 
infers the entire subteam cannot perform its role. 

If, based on the role-monitoring constraints and the role performance information about 
teammates, STEAM infers team operator [OP]e to be unachievable, it invokes [repair]e for 
replanning. By casting repair as a team operator, agents automatically ensure the entire 
team's commitment for their replanning (the entire team is affected if [OP]e is unachiev- 
able). Furthermore, agents inform teammates not only about possible repair results, but 
also repair unachievability or irrelevancy. The actions taken in service of [repair]e depend 
on the context. If [repair]e was invoked due to [OP]e's domain-specific unachievability 
conditions, domain-specific repair is triggered. In contrast, if [repair]e was invoked due to 
role-monitoring constraint failures, STEAM leads each agent to first analyze the failure. 
The analysis may reveal a critical role failure — a single role failure causing the unachiev- 
ability of [OP]e — which may occur in an AND-combination if any agent or subteam fails 
in its role; or an OR- combination when all team members are role-dependent on a sin- 
gle individual or a single subteam. For instance, when agents are fiying in formation via 
[travelling]e (OR-combination), everyone is role-dependent on the lead helicopter. Thus, 
should the lead crash, a critical role failure occurs. 

The action taken in cases of a critical role failure is team reconfiguration, to determine 
a team member, or subteam, to substitute for the critical role. As mentioned earlier, this 
situation corresponds to the "unreconciled case" in SharedPlans, discussed in Section 3.2. 
The steps taken in STEAM in this case are as follows: 

1. Determine candidates for substitution: Each team member first matches it own capabilities 
or those of other agents or subteams with the requirements of the critical role. Matching 
currently relies on domain-specific knowledge. Of course, agents or subteams that are the 
cause of the critical role failure cannot be candidates for substitution. 

2. Check for critical conflicting commitments: Once an agent determines possible candidate(s), 
including itself, it checks for conflicts with candidate's existing commitments to the team. If 
these commitments are already critical, the candidate is eliminated from consideration. For 
instance, if the candidate is a participant in a team operator which is an AND-combination, 
its responsibilities to the team are already critical — even if it possesses relevant capabilities, 
it cannot take over the role in question. Similarly, the candidate is ruled out if all other team 
members are role-dependent on it. 

3. Announce role- substitution to the team: Candidate(s) not ruled out in step 2 can substitute 
for the role. This could mean an individual volunteering itself, or a team leader volunteering 
its subteam for the critical role. Since [repair]© is a team operator, and since role-substitution 
implies its achievement, any role-substitution is announced to 0. 
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4. Delete non-cnttcal conflicting commitments: After assuming the new role in the team activity, 
the relevant individual or subteam members delete their old roles and old commitments. 

In the Attack domain, team members can follow the above procedure when recovering 
from critical role failures such as item 5 in Figure 5. There, since a scouting subteam is 
not specified, and the relevant operator wait-while-hattle-position-scouted involves an AND- 
combination of the scouting role with the non-scouts, a critical role failure occurs. A 
subteam in the rest of the company is located to possess the capabilities of scouting. The 
leader of this subteam determines that it can volunteer its subteam for scouting, and an- 
nounces this change in role to the rest of the team. Members of this subteam then delete 
confiicting commitments. [wait-while-battle-position-scouted]e is now executed with this 
new role assignment. (Since such new role assignments are confined to the local context of 
individual team operators, and since step 2 explicitly checks for critical confiicts, they do 
not lead to any global side-effects.) 

The entire repair procedure above can invoked in the context of a subteam i7, rather than 
the team 0. In this case, [repairjo will be invoked as a team operator. STEAM follows 
an identical repair procedure, in this case enabling individuals or sub-subteams to take 
over particular critical roles. Furthermore, any repair communication here is automatically 
restricted within i7. 

In case the failure is a pure role dependency failure, only a single dependent agent z/i 
is disabled from role performance (because opj^j- =^ opj,j). Here, z/i must locate another 
agent z/k such that opj^j- =^ opj.^. Role dependency failure could involve a subteam fig- 
instead of an individual; and the subteams engage in an identical repair. 

If failure type is all roles failure, no agent performs its role; this state is irreparable. 
In this situation, or in case no substitution is available for a critical role, [repair]e is itself 
unachievable. Since the repair of [OP]e is itself unachievable, a complete failure is assumed, 
and [complete-failure] e is now invoked. For instance, in the Attack domain, complete 
failure implies returning to home base. By casting complete-failure as a team operator, 
STEAM ensures that team members will not execute such drastic actions without consulting 
teammates. If only a subteam i7 or an individual z/i encounters complete-failure, they infer 
inability to perform their roles in the team 0's on-going activity. 

5. STEAM: Selective Communication 

STEAM agents communicate to establish and terminate team operators. Given the large 
number of team operators in a dynamic environment, this communication is a very sig- 
nificant overhead (as Section 6 shows empirically), or risk (e.g., in hostile environments). 
Therefore, STEAM integrates decision-theoretic communication selectivity. Here, STEAM 
takes into consideration communication costs and benefits, as well as the likelihood that some 
relevant information may he already mutually believed. While this pragmatic approach is 
a response to the constraints of real-world domains, it is not necessarily a violation of the 
prescriptions of the joint intentions framework. In particular, the joint intentions frame- 
work does not mandate communication, but rather a commitment to attain mutual belief. 
Via its decision-theoretic communication selectivity, STEAM attempts to follow the most 
cost-effective method of attaining mutual belief relevant in joint intentions. 
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Figure 7 presents the decision tree for the decision to communicate a fact F, indicating 
the termination of [OP]e- Rewards and costs are measured to the team, not an individual. 
The two possible actions are NC (not communicate, cost 0) or C (communicate, cost Cc). 
If the action is NC, two outcomes are possible. With probability (1-t), F was commonly 
known anyway, and the team is rewarded B for terminating [OP]e- With probability r, 
however, F was not known, and thus there is miscoordination in terminating [OP]e (e.g., 
some agents come to know of F only later). Given a penalty Cmt for miscoordination, the 
reward reduces to B-C'mt- If the action is C, assuming reliable communication, F is known. 

Rewards 
(F known) B 
(F unknown) b - Cmt 

(F known) B 

(F unknown) B - Cmt 

Figure 7: Decision tree for communication. 

EU(C), the expected utility of option C, is B-Cc . EU(NC) of option NC is B-T*Cmt- 
To maximize expected utility, an agent communicates iff EU(C) > EU(NC), i.e., iff: 

T * C'mt > Cc 

Thus, for instance, in the Attack domain, when flying with high visibility, pilot agents do 
not inform others of achievement of waypoints on their route, since r is low (high likelihood 
of common knowledge), and Cmt is low (low penalty). However, they inform others about 
enemy tanks on the route, since although r is low, Cmt is high. The communication cost 
Cc could vary depending on the situation as well, and team members may flexibly reduce 
(increase) communication if the cost increases (decreases) during their team activity. Inter- 
estingly, if only a single agent is left in a team, r drops to zero, and thus, no communication 
is necessary. 

Expected utility maximization is also used for selectivity in the establish commitments 
protocol. If 7 is the probability of lack of joint commitments, and Cme the penalty for 
executing [OP]e without joint commitments from the team, then an agent communicates 
iff EU(C) > EU(NC), i.e., iff: 

7 * Cme > Cc 

5.1 Further Communication Generalization 

Further generalization in communication is required to handle uncertainty in the termina- 
tion criteria for joint intentions. For instance, a team member z/4 may be uncertain that an 
enemy tank seen enroute causes [fly-flight-plan] e to be unachievable — the tank's threat 
may not be clearcut. Yet not communicating could be highly risky. The decision tree for 
communication is therefore extended to include S, the uncertainty of an event's threat to 
the joint intention (Figure 8). Since agents may now erroneously inform teammates to 
terminate team operators, a nuisance cost -Cn is introduced. 
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Rewards 

(Terminates) B 
) (Not Terminate) 
(Terminates) B-Cmt 
(Not Terminate) q 

(Terminates) B 
(Not Terminate) -Cn 

Figure 8: Extended decision tree with 6. 

Again, an agent communicates iff EU(C) > EU(NC), i.e., iff ^*r*Cmt > (Cc + (l-^)Cn). 
If 6 is 1, i.e., a team operator has terminated, this equation reduces to — T*Cmt > Cc — 
seen previously. If ^ << 1, i.e., high uncertainty about termination, no communication 
results if Cn is high. Therefore, the decision tree is further extended to include a new 
message type — threat to joint intention — where Cn is zero, but benefits accrued are 
lower (B - C^). This threat message maximizes expected utility when 6 << 1, i.e., if Cn is 
high for communicating termination, a team member communicates a threat. For instance, 
a threat message is used if an agent fails in its own role, which is a threat to the joint 
intention. However, as before, termination messages are used when ^ = 1, where they 
maximize expected utility. 

5.2 Estimating Parameters (7, r, 6) 

As a first step, STEAM only uses qualitative (low, high, medium) parameter values. STEAM 
estimates likelihood of lack of joint commitments 7, via team tracking (Tambe, 1996) — 
dynamically inferring a team's mental state from observations of team members' actions. 
Fortunately, rather than tracking each teammate separately, an agent z/i can rely on its own 
team operator execution for team tracking. In particular, suppose z/i has selected a team 
operator OP for execution, and it needs to estimate 7 for operator OP, and its team 0. 
Now, if z/i selected OP at random from a choice of equally preferable candidates, then its 
teammates may differ in this selection. Thus, there is clearly a low likelihood of a joint 
commitment — z/i estimates 7 to be high. However, if OP is the only choice available, then 
7 depends on the preceding [0P2]q that z/i executed with the team i7. (i7 may be just a 
singleton, i.e., 0P2 may be an individual operator that z/i executed alone). There are three 
cases to consider. First, if C J) (0 is subteam of i7) or = i7, all members of were 
jointly executing [0P2]q. Furthermore, [0P2]q could only be terminated via mutual belief 
among 0. Thus, is likely to be jointly committed to executing the only next choice OP — 
7 is estimated low. Second, if i7 C 0, some members in were not jointly participating in 
team operator execution earlier; hence 7 is estimated high. Third, if no operator precedes 
OP, e.g., OP is first in a subgoal, then 7 is estimated low. 

While agents usually infer matching estimates of 7, sometimes, estimates do mismatch. 
Therefore, STEAM integrates some error recovery routines. For instance, if an agent z/i 
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estimates 7 to be low, when others estimate it high, z/i starts executing the team operator, 
and only later receives messages for establishing joint commitments, z/i recovers by stopping 
current activities and re-establishing commitments. In contrast, if z/i mis-estimates 7 to be 
high, it unnecessarily waits for messages for establishing commitments. STEAM infers such 
a mis-estimation via reception of unexpected messages; it then conducts a lookahead search 
to catch up with teammates. 

To estimate r (the probability that a fact is not common knowledge), STEAM assumes 
identical sensory capabilities for team members, e.g., if some fact is visible to an agent, 
then it is also visible to all colocated teammates. However, at present, domain knowledge 
is also required to model information media such as radio channels, in estimating r. S, 
the probability of an event's threat to a joint intention, is estimated 1 if a fact matches 
specified termination conditions. Otherwise role monitoring constraints are used, e.g., in 
an OR- combination, S is inversely proportional to the number of team members. The cost 
parameters, Cmt, Cme, and Cc are assumed to be domain knowledge. 

6. Evaluation 

STEAM is currently implemented within Soar via conventions for encoding operators and 
states, plus a set of 283 rules. Essentially these rules help encode the algorithm in Ap- 
pendix A; some sample rules are presented in Appendix B. STEAM has been applied in the 
three domains mentioned earlier: Attack, Transport and RoboCup. Table 1 provides some 
information about the three domains. Column 1 lists the three domains. Column 2 lists 
the maximum number of agents per team in each domain. Column 3 shows the possible 
variations in the sizes of the team. Thus, in the Attack and Transport domains, the team 
sizes may vary substantially; but not so in RoboCup. Column 4 shows the number of levels 
in the team organization hierarchy (e.g., the team-subteam-individual hierarchy is a three 
level hierarchy). Column 5 shows the maximum number of subteams active at any one time. 



Domain 


Max team 


Team size 


Levels in 


Maximum num 


name 


size 


varation 


team hierarchy 


subteams 


Attack 


8 


2-8 


3 


2 


Transport 


16 


3-16 


4 


5 


RoboCup 


11 


11 


3 


4 



Table 1: The organizational hierarchy in the three domains. 

steam's application in these three domains provides some evidence of its generality. 
In particular, not only do these domains differ in the team tasks performed, but as Table 
1 illustrates, the domains differ substantially in the team sizes and structure. The rest of 
this section now uses the three domains in detailed evaluation of STEAM using the criteria 
of overall performance, reusability, teamwork fiexibility, communication efficiency, as well 
as effort in encoding and modifying teamwork capabilities. 
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6.1 Overall Performance 

One key evaluation criterion is the overall agent-team performance in our three domains. 
Ultimately, STEAM-based agent teams must successfully accomplish their tasks, within 
their given environments, both efficiently and accurately. This is a difficult challenge in 
all three domains. Certainly, the Attack and Transport domains involve complex synthetic 
military exercises with hundreds of other agents. Furthermore, in these domains, the domain 
experts (expert human pilots) define the pilot teams' missions (tasks), rather than the 
developers. STEAM-based pilot teams have so far successfully met the challenges in these 
domains — they have successfully participated in not one, but about 10 such synthetic 
exercises, where the domain experts have issued favorable written and verbal performance 
evaluations. 

In the RoboCup domain, our player team must compete effectively against teams devel- 
oped by other researchers worldwide. At the time of writing this article, our player team 
easily wins against the winner of the pre-RoboCup'96 competition. However, all teams con- 
tinue to evolve, and researchers continue to field new sophisticated teams. One key test for 
all the teams in the near future is the RoboCup'97 tournament at the International Joint 
Conference on Artificial Intelligence (IJCAI), Nagoya, Japan, in August 1997. 

6.2 Reuse of Teamwork Capabilities 

steam's inter-domain and intra-domain reusability is approximately measured in Table 
2. Column 1 once again lists the three different domains of STEAM's application. Column 
2 lists the total number of rules per agent per domain — which include the rules that 
encode the domain knowledge acquired from domain experts as well as STEAM rules — 
illustrating complexity of the agents' knowledge base. The number of STEAM rules used 
in these domains is listed in Column 3. Column 4 measures percent reuse of STEAM rules 
across domains. (No reuse is shown in STEAM's first domain. Attack). There is 100% 
reuse in Transport, i.e., no new coordination/communication rules were written — a major 
saving in encoding this domain. RoboCup, in its initial stages, has lower reuse. Here, due 
to weakness in spatial reasoning and tracking, agents fail to recognize other team's play, or 
even own teammates' failures (e.g., in executing a pass), hampering the reuse of rules for 
role-monitoring constraints, repair and threat detection. With improved spatial reasoning 
and tracking, reuse may improve in the future. Column 5 lists the total number of team 
operators specified per domain, illustrating significant intra-domain reuse — essentially, for 
each team operator, STEAM's entire teamwork capabilities are brought to bear. 



Domain 


Total rules 


STEAM rules 


STEAM reuse 


Team operators 


Attack 


1575 


283 


first-use 


17 


Transport 


1333 


283 


100% 


14 


RoboCup 


454 


110 


38% 


11 



Table 2: STEAM reusability data. 
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6.3 Flexibility in Teamwork 

Teamwork flexibility is closely related with the measures of overall performance and reusabil- 
ity. Since STEAM's entire teamwork capabilities are brought to bear in executing team 
operators in all of the domains, there are signiflcant improvements in teamwork flexibility. 
For instance, in benchmark runs of Attack, almost all of the teamwork failures from our 
earlier implementation are avoided. Certainly, all of the failures in Figure 5 are addressed: 

• Items 1 and 7 are addressed because agents must now attain mutual belief in the 
achievement, unachievability or irrelevancy of team operators. Thus, in item 1, the 
commander now attains mutual belief that the helicopter company has completed its 
engagement with the enemy; while in item 7, the irrelevancy of planning a bypass 
route is communicated to the company. 

• Items 3 and 6 are addressed because agents now act jointly by flrst ensuring the 
establishment of joint commitments before executing their roles. For instance, a team 
member does not begin executing the mission as soon as it processes its orders (item 
3); rather, it acts jointly with the team, after the team establishes joint commitments 
to execute the mission. 

• Items 2, 4 and 5 are addressed because the team operator wait-while-hattle-position- 
scouted is specifled to be an AND-combination of the role of the scouts and the 
non-scouts. Thus, unachievability of team operators is detected, since either the 
scouts or the non-scouts cannot perform their role, or the scouting-role assignment is 
unspecifled. In items 2 and 4 no repairs are possible, but at the least the company 
infers a "complete-failure" and returns to home base, instead of waiting indeflnitely. 
In item 5, the unassigned role again leads to unachievability, but repair is possible 
because one of the remaining subteams can take over the role of the scout. 

• Item 8 is addressed since the relevant operator engage is now explicitly deflned as a 
team operator with an OR- combination of members' roles. Thus, based on commu- 
nication from team members, team members' can infer its unachievability. 

• Item 9 is addressed because in the establish-commitments protocol, the leader will 
repeat its message if a response is not heard within time limit. However, in general, 
attaining mutual belief given the possibility of uncertain communication channels is 
a notoriously difficult challenge (Halpern & Moses, 1990); and this remains an issue 
for future work. 

As a further illustration of teamwork flexibility in STEAM, we created six variations in 
the environmental conditions facing the Attack company of helicopter pilots. Each condition 
required the pilot team to flexibly modify its communication to maintain coherence in 
teamwork. The six variations are: 

1. Condition 1: This is the baseline "normal" condition. 

2. Condition 2: Although similar to condition 1, we assume in addition that certain radio 
frequencies /channels which were previously separated, are now common. In partic- 
ular, messages previously assumed to be privately delivered to only the commander 
agent from its superiors, are now also made available to the other team members. 
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3. Condition 3: Although similar to condition 2, the communication cost is raised from 
"low" to "medium". 

4. Condition 4- Although similar to condition 3, we assume in addition that the he- 
licopter team has only a medium priority for ensuring simultaneous attack on the 
enemy. 

5. Condition 5: Here, we once again start with the baseline of condition 1, but assume 
poor visibility in addition. Thus, agents may not accurately estimate their distances. 

6. Condition 6: In addition to condition 5, here, the company has some flexibility in 
reaching the battle position. The company is provided with the option of halting at 
certain key locations, rather than continuing to fly. 

The decision-theoretic framework in STEAM enables agents to flexibly respond to the 
above conditions. Figure 9 plots the number of messages exchanged among team members 
for each of the six conditions. The total number of messages in three teams — balanced, 
cautious and reckless — are compared. Balanced agents fully exploit the decision theory 
framework, and thus illustrate STEAM's flexibility. Cautious agents always communicate, 
ignoring the decision theory framework. Reckless agents communicate very little (only if 
high Cmt, Cme)- Of course, truly reckless agents would likely not communicate at all, so 
this definition is relaxed here. All three teams work with identical cost models, Cc, Cmt, 
and Cme- The number of agents were fixed in this experiment to four, so all three teams 
— cautious, balanced and reckless — could be run (as discussed in the next section, it is 
difficult to run the cautious team with further increase in team size). 

Focusing first on the balanced team, it was able to perform its mission under all six 
conditions, by fiexibly decreasing or increasing the number of messages in response. The 
first set of conditions (conditions 2 through 4) illustrate that the balanced team can reduce 
its communication in response to the situation faced, e.g., increase in communication cost. 
However, under conditions 5 and 6, the balanced team can also increase its communication 
to address the uncertainties. For instance, with condition 5, knowledge of poor visibility 
automatically leads team members to explicitly communicate achievement of waypoints 
on their route. In addition, with condition 6, the team has to communicate to establish 
commitments when deciding to halt or to fiy forward. 

The cautious team was also able to perform the mission under all six conditions, but it 
relies on many more messages and remains insensitive to conditions 2-4 that should result 
in fewer messages. Indeed, its exchange of 10-20 fold more messages than the balanced 
team to perform an identical task is not only a waste of precious communication resources, 
but can create risks for the team in hostile environments. (The next subsection will discuss 
the issue of communication efficiency in more detail.) The reckless team does communicate 
fewer messages, but it fails to perform its basic mission. Even in the first normal case, 
this helicopter company gets stuck on the way to the battle position, since a message with 
medium Cmt but high r is not communicated. Interestingly, the number of messages increase 
in the reckless team under conditions 2-4. This is because condition 2 allows the reckless 
team to avoid getting stuck before reaching the battle position. (In fact, this condition was 
designed to get the reckless team unstuck.) Since the reckless team can now perform more of 
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Figure 9: Change in communication with additional uncertainties. 



the mission — reaching its battle position — more messages are exchanged. Unfortunately, 
some key messages are still not exchanged, leaving team members stranded in the battle 
position. 

6.4 Communication Efficiency 

Communication efficiency is critical in teamwork, particularly with scale-up in team size, else 
communication overheads can cause significant degradation in team performance. Figure 
10 and 11 attempt to measure the communication overhead of teamwork, and the usefulness 
of steam's decision-theoretic communication selectivity in lowering the overhead, partic- 
ularly for a scale-up in team size. Both the figures compare the total number of messages 
in the three teams introduced above — balanced, cautious and reckless — with increasing 
numbers of agents per team. In the interest of a fair comparison, the total computational 
resources available to each team were kept constant (a single SUN Ultral). While this 
limits the maximum team size that could be run, the results shown below are sufficiently 
illustrative in terms of scale-up. 

Figure 10 focuses on the Attack domain. Decision-theoretic selectivity enables the bal- 
anced team to perform well with few messages — this team is regularly fielded in synthetic 
exercises. The cautious team exchanges 10 to 20-fold or more messages than the balanced 
team — a substantial communication overhead. Indeed, beyond six agents, the simulation 
with cautious team could not be run in real time.'' Reckless agents in this case do not 
exchange any messages at all. 

Figure 11 focuses on the Transport domain; once again comparing the performance of 
cautious, balanced and reckless teams for increasing numbers of agents in the team. Once 
again, decision-theoretic selectivity enables the balanced team to perform well with few 
messages — this team is regularly fielded in synthetic exercises. The cautious team once 
again incurs a significant overhead of 10 to 20-fold or more messages than the balanced 

7. The earlier experiments in Section 6.3 were run with four agents per team, so that the cautious team 
could be run in real-time. 
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Figure 10: Attack domain: selective communication. Reckless team exchanges no messages 
and hence that plot overlaps with the x-axis. 



team. Here, beyond seven agents, the simulation with the cautious team could not be 
run in real time. Interestingly, in the test scenario for this experiment, the reckless team 
is able to perform the mission appropriately even though this team exchanges just 1-2 
messages, far fewer than the balanced team. To a certain extent, this result illustrates the 
potential for improving the decision-theoretic selectivity in the balanced team. However, 
when the test scenario for this experiment was changed, so that the transports arrived late 
at the rendezvous point, the balanced team was able to continue to perform the mission 
appropriately. However, the reckless team now performed inappropriately, highlighting the 
risk in the reckless approach. 
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Figure 11: Transport domain: selective communication. 

Figure 12 illustrates the differing communication patterns in the cautious and balanced 
teams for the Attack domain, to attempt to understand the difference in their total com- 
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munication. Figure 12-a plots the varying degree of collaboration (y-axis) during different 
phases (x-axis) in the Attack domain. Degree of collaboration is measured as the percentage 
of team operators in a pilot's operator hierarchy (which consists of team and individual op- 
erators). A low percentage implies low degree of collaboration and vice versa. The solid line 
plots the overall degree of collaboration in the team, taking into account all team operators. 
The dashed line indicates the degree of collaboration without counting team operators exe- 
cuted by this pilot agent's subteam — the differing pattern in the two lines is an indication 
of the differing degree of subteam activity. In particular, the two lines sometimes overlap 
but separate out at other times, indicating the flexibility available to the subteam. The 
overall degree of collaboration is lowest in phases 18-20 (20-40%), where agents engage the 
enemy. Figure 12-b plots the percentage of total communication per phase, for cautious 
and balanced teams. For instance, the cautious team exchanges 1% of its total messages in 
phase 20. Communication percentage is correlated to the degree of collaboration per phase 
for the cautious team (coefficient 0.80), but not for the balanced team (coefficient -0.34). 
Essentially, unlike the cautious team, the balanced team does not communicate while their 
collaboration proceeds smoothly. 
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Figure 12: Attack domain: pattern of communication 



6.5 Encoding and Modification Effort 

The final evaluation criteria focus on the effort involved in encoding and modifying agents' 
teamwork capabilities — comparing the effort in STEAM with alternatives. The key alter- 
native is reproducing all of STEAM's capabilities via special-case coordination plans, as in 
our initial implementation in the Attack domain. We estimate that such an effort would 
require significant additional encoding effort. For example, just to reproduce STEAM's 
selective communication capabilities, our initial implementation could potentially have re- 
quired hundreds of special case operators. Consider our initial implementation in the Attack 
domain. Here, the 17 team operators in STEAM (which would only be individual operators 
in the initial implementation), would each require separate communication operators — two 
operators each to signal commitments (request and confirm) and one to signal termination 
of commitments. That is already a total of 51 (17x3). Furthermore, to reproduce selectivity, 
additional special cases would be necessitated — in the extreme case, each combination of 
values of (r, Cmt, and Cc) or (7, Cme, and Cc), could require a separate special case operator 
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(51 X total combinations, already more than a hundred). Furthermore, separate operators 
may be required depending on whether the communication occurs with the entire team or 
only a subteam. Of course, it would appear that all such special cases could be economized 
in our initial implementation by discovering generalizations — but then STEAM encodes 
precisely such generalizations to avoid the many special cases. 

An additional point of evaluation is easy of modifiability of agent team behaviors. In 
our experience, domain knowledge acquired from experts is not static — rather it undergoes 
a slow evolution. In the Attack domain, for instance, real-world military doctrine continues 
to evolve, requiring modifications in our synthetic pilot team behaviors. In such situations, 
STEAM appears to facilitate such modifications suggested by domain experts; at least, it 
is often not necessary to add new coordination plans. For instance, in the Attack domain, 
domain experts earlier suggested a modification, that the helicopter company should evade 
enemy vehicles seen enroute, rather than fiying over. Here, adding a new unachievabil- 
ity condition for the team operator [fiy-fiight-plan]e was sufficient; STEAM then ensured 
that the pilot agents coordinated the termination of [fiy-fiight-plan]e , even if just one arbi- 
trary team member detected the enemy vehicles. (Of course, the evasion maneuvers, being 
domain-specific, had to be added.) 

7. Related Work 

As mentioned earlier, most implementations of multi-agent collaboration continue to rely on 
domain-specific coordination in service of teamwork (Jennings, 1994, 1995). More recently, 
however, a few encouraging exceptions have emerged (Jennings, 1995; Rich & Sidner, 1997). 
We first briefiy review these systems and then contrast them with STEAM. 

Jennings's (1995) implementation of multi- agent collaboration in the domain of electric- 
ity transportation management is also based on joint intentions — it is likely one of the first 
implementations in a complex domain based on a general model of teamwork. He presents 
a framework called joint responsibility based on a joint commitment to the team's joint 
goal a and a joint recipe commitment to a common recipe S. Two distinct types of joint 
commitments — a modification to the joint intentions framework — are claimed necessary 
because different actions are invoked when joint commitments are dropped. However, as 
a result, joint responsibility would appear to be limited to a two-level hierarchy of a joint 
goal and a joint plan, although individuals could execute complex activities in service of 
the joint plan. The joint responsibility framework is implemented in the GRATE* system, 
which appears to focus on a team of three agents. In GRATE*, teamwork proceeds with an 
organizer agent detecting the need for joint action; it is then responsible for establishing a 
team and ensuring members' commitments as required by the joint responsibility method. 
While the procedure for establishing joint commitments in STEAM is similar to GRATE* 
— including the similarity of the "leader" in STEAM to the "organizer" in GRATE* — 
STEAM does benefit from adopting PWAGs, which provides it additional fiexibility. 

STEAM is also related to COLLAGEN (Rich & Sidner, 1997), a prototype toolkit ap- 
plied to build a collaborative interface agent for applications such as air-travel arrangements. 
COLLAGEN's origins are in the SharedPlans theory. Although the COLLAGEN implemen- 
tation does not explicitly reason from the intend that attitude in SharedPlans introduced 
in (Grosz & Kraus, 1996), it does incorporate discourse generation and interpretation algo- 
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rithms that originate in such reasoning (Lochbaum, 1994). Treating the underlying agent as 
a blackbox, COLLAGEN facilitates the discourse between a human user and the blackbox 
(intelligent agent). Several COLLAGEN features aid in such interaction, such as mainte- 
nance of a segmented interaction history. 

STEAM contrasts with COLLAGEN (Rich & Sidner, 1997) and GRATE* (Jennings, 
1995) in several important ways. First, STEAM builds on joint intentions (with some 
influence of SharedPlans), rather than the SharedPlan approach in COLLAGEN or the 
joint responsibility approach of GRATE*. Particularly in contrast with joint responsibility, 
STEAM allows teamwork based on deep joint goal/plan hierarchies. Second, STEAM has 
the capability for role-monitoring constraints and role substitution in repairing team activi- 
ties, not relevant in the other two systems. Third, STEAM has attempted scale-up in team 
size. Thus, STEAM has introduced techniques both to reduce teamwork overheads, e.g., 
decision-theoretic communication selectivity, as well as to deal with a hierarchy of teams 
and subteams, not relevant in smaller-scale teams. STEAM also illustrates reuse across 
domains, not seen in the other two systems. Finally, rather than building a collaboration 
layer on top of an existing domain-level system or blackbox ("loose coupling"), STEAM has 
proposed tighter coupling via modifications to support teamwork in the agent architecture 
itself, e.g., with explicit team goals and team states, and accompanying commitments. The 
determining factor here would appear to be the tightness of collaboration, e.g., a deeply 
nested, dynamic joint goal hierarchy should favor a tighter coupling. 

In our previous work (Tambe, 1997b) we presented an initial implementation of a team- 
work model, also based on joint intentions. That work clearly laid the groundwork for 
STEAM, by defining team operators, and elaborating on their expressiveness. However, 
STEAM was later developed because of (i) several problems in that work in continued de- 
velopment of teamwork capabilities in the Attack domain, (ii) the presence of new domains 
such as Transport, and (iii) significant scale-up in team sizes. Since STEAM both extends 
and substantially revises that earlier work, it is best to treat STEAM as a separate system, 
rather than an extension of that early work. STEAM also provides a conceptual advance 
in a clearer analysis and specification of the joint mental attitude it builds up in a team. 
In particular, via an explicit analogy to partial SharedPlans (Grosz & Kraus, 1996), this 
article has spelled out the requirement for teams and subteams to build up a hierarchy of 
joint intentions, beliefs about other team members' intentions, and joint intentions for the 
"unreconciled case" . This analysis also led to a generalization of communication based on 
information-dependency. 

The following now presents a detailed comparison between STEAM and the earlier work 
(Tambe, 1997b) in terms of their capabilities. To begin with, STEAM includes an explicit 
mechanism to establish joint commitments based on PWAGs, which was unaddressed in 
previous work — so earlier, agents would implicitly, and hence sometimes incorrectly, as- 
sume the existence of joint commitments. Also, in earlier work, monitoring and repair 
was highly specialized. In particular, the mechanism provided for monitoring was based 
on comparing achievement conditions of operators; this mechanism was later discovered 
to be limited to monitoring and repair of just one pre-determined specialist role per team 
operator. Furthermore, the role-substitution was defined via a special procedure executed 
separately by individuals. In contrast, STEAM has significantly generalized monitoring and 
repair via its explicit role-monitoring constraints, that enable monitoring of a much greater 
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variety of failures, e.g., the "specialist" is just one case in all of the varied role- monitoring 
constraint combinations. Furthermore, STEAM establishes a joint intention to resolve all 
failures, rather than relying on any special case procedures. This is not merely a conceptual 
advance in terms of an integrated treatment of repair, but has real behavioral implications 
in providing additional flexibility embodied in the commitments in [repair]e. Further- 
more, steam's repair generalizes to subteams, addresses previous critical commitments, 
as well as unallocated tasks. In terms of practical concerns, our previous work (Tambe, 
1997b) raised the issue of communication risk in hostile environments, but suggested only a 
heuristic evaluation of communication costs and benefits; a general purpose mechanism was 
lacking. STEAM has filled the gap with its decision theoretic framework that now considers 
various uncertainties, both for selective communication as well as enhancements in commu- 
nication. Also, unlike STEAM, our earlier work did not deal with complex team-subteam 
hierarchies, and its mechanisms did not generalize to subteams. Finally, STEAM is backed 
up with detailed experimental results about both its fiexibility and reuse across domains, all 
outside the scope of the previous work. 

STEAM is also related to coordination frameworks such as Partial Global Planmng(F GF) 
(Durfee & Lesser, 1991), and Generalized Partial Global Planning(GFGF) (Decker & Lesser, 
1995). Although not driven via theories of collaboration, these coordination frameworks also 
strive towards domain independence. The earlier work on PGP focuses on a system of co- 
operating agents for consistent interpretation of data from a distributed sensor network 
(Durfee & Lesser, 1991). Here, subordinate agents may exchange their individual goals 
and plans of action. An assigned agent (e.g., a supervisor) may recognize that individual 
plans of different agents meld into a partial global plan (PGP) — so called because PGPs 
involve more than one agent but not necessarily all agents (partially global) — in service 
of a common group goal. The PGP is a basis for planning coordination actions; and it 
may be transmitted to subordinates for guidance in execution of individual actions. (PGP 
can accommodate different types of organizations as well.) GPGP (Decker & Lesser, 1995) 
provides several independent coordination modules, any subset of which may be combined 
in response to coordination needs of a task environment; the GPGP approach can duplicate 
and extend the PGP algorithm. 

As a general model of teamwork, STEAM can provide a principled underlying model to 
reason about at least some of the coordination specified in PGP, e.g., agents would establish 
a joint intention towards the collective goal in a PGP, and modulate their communication 
via decision-theoretic reasoning. That is, PGP "compiles out" some of the underlying rea- 
soning in STEAM, and thus STEAM could provide additional fiexibility in coordination. 
Essentially, PGP and GPGP do not separate out coordination in teamwork from coordina- 
tion in general (such as via a centralized coordinator). As a result, they fail to exploit the 
responsibilities and commitments of teamwork in building up coordination relationships. 
Analogously, some of the general coordination in PGP or GPGP is unaccounted for in 
STEAM, and hence understanding relationships among STEAM and GPGP is an interest- 
ing area of future work. There is a similar relationship between STEAM and the COOL 
coordination framework (Barbuceanu & Fox, 1996). COOL also focuses on general pur- 
pose coordination by relying on notions of obligations among agents. However, it explicitly 
rejects the notion of joint goals and joint commitments. It would appear that individual 
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commitments in COOL would be inadequate in addressing some teamwork phenomena, but 
further work is necessary in understanding the relationship among COOL and STEAM. 

In team tracking (Tambe, 1996), i.e., inferring team's joint intentions, the expressiveness 
of team operators has been exploited. However, issues of establishing joint commitments, 
communication, monitoring and repair are not addressed. The formal approach to team- 
work in (Sonenberg et. al., 1994) transforms team plans into separate role-plans for exe- 
cution by individuals, with rigidly embedded communications. STEAM purposely avoids 
such transformations, so agents can flexibly reason with (i) explicit team goals/plans; and 
(ii) selective communication (seen to be important in practice). In (Gmytrasiewicz, Dur- 
fee, & Wehe, 1991), decision theory is applied for message prioritization in coordination 
based on the agents' recursive modeling of each others' actions. STEAM applies decision 
theory for communication selectivity and enhancements, but in a very different context — 
practical operationalization of general, domain-independent teamwork model based on joint 
intentions. 

8. Summary and Future Work 

Teamwork is becoming increasingly critical in a variety of multi-agent environments, rang- 
ing from virtual environments for training and education, to internet-based information 
integration, to potential multi-robotic space missions (Tambe et al., 1995; Rao et al., 1993; 
Pimentel & Teixeira, 1994; Williamson et al., 1996; Kitano et al., 1997; Hayes-Roth et al., 
1995; Reilly, 1996). In previous implementations of multi-agent systems, including our own, 
teamwork has often been based on pre-defined, domain-specific plans for coordination. Un- 
fortunately, these plans are infiexible and thus no match for the uncertainties of complex, 
dynamic environments. As a result, agents' coherent teamwork can quickly dissolve into 
miscoordinated misbehavior. Furthermore, the coordination plans cannot be reused in other 
domains. Such reuse is important however, both to save implementation effort and enforce 
consistency across applications. 

Motivated by the critical need for teamwork fiexibility and reusability, this article has 
presented STEAM, a general model of teamwork. While STEAM's development is driven 
by practical needs of teamwork applications, its core is based on principled theories of team- 
work. STEAM is one of just a few implemented systems that have begun to bridge the gap 
between collaboration theories and practice. STEAM combines several key novel features: 
(i) use of joint intentions as a building block for a team's joint mental attitude (Levesque 
et al., 1990; Cohen & Levesque, 1991b) — the article illustrates that STEAM builds up 
a hierarchical structure of joint intentions and individual intentions, analogous to the par- 
tial SharedPlans (Grosz & Kraus, 1996); (ii) integration of novel techniques for explicit 
establishment of joint intentions (Smith & Cohen, 1996); (iii) principled communication 
based on commitments in joint intentions; (iv) use of explicit role-monitoring constraints 
as well as repair methods based on joint intentions; (v) application of decision-theoretic 
techniques for communication selectivity and enhancements, within the context of the joint 
intentions framework. To avail of the power of a model such as STEAM, a fundamental 
change in agent architectures is essential — architectures must provide explicit support for 
representation of and reasoning with team goals, (reactive) team plans and team states. 
STEAM has been applied and evaluated in three complex domains. Two of the domains. 
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Attack and Transport, are based on a real- world simulation environment for training, and 
here our pilot agent teams have participated large-scale synthetic exercises with hundreds of 
other synthetic agents. In the third domain, RoboCup, our player agent team is now under 
development for participation in the forthcoming series of (simulated) soccer tournaments, 
beginning at IJCAI-97. 

Of course, STEAM is far from a complete model of teamwork, and several major is- 
sues remain open for future work. One key issue is investigating STEAM's interactions 
with learning. Initial experiments with chunking (Newell, 1990) (a form of explanation- 
based learning (Mitchell, Keller, & Kedar-Cabelli, 1986)) in STEAM reveal that agents 
could automatize routine teamwork activities, rather than always reasoning about them. 
Specifically, from STEAM's domain-independent reasoning about teamwork, agents learn 
situation-specific coordination rules. For instance, when the formation leader crashes, an- 
other agent learns situation-specific rules to take over as formation lead and communicate. A 
well-practiced team member could thus mostly rely on learned rules for "routine" activities, 
but fall back on STEAM rules if it encounters any unanticipated situations. Additionally, 
STEAM's knowledge-intensive to learning approach could complement current inductive 
learning approaches for multi-agent coordination (Sen, 1996). 

Failure detection and recovery is also a key topic for future work, particularly in environ- 
ments with unreliable communication. One novel approach exploits agent tracking (Tambe 
& Rosenbloom, 1995; Tambe, 1996) to infer teammates' high-level goals and intentions for 
comparison with own goals and intentions. Differences in goals and intentions may indicate 
coordination failures, since teammates often carry out identical or related tasks. However, 
given the overheads of such an approach, it has to be carefully balanced with an agents' 
other routine activities. Initial results of this approach are reported in (Kaminka & Tambe, 
1997). 

Enriching STEAM's communication capabilities in a principled fashion is yet another 
key topic for future work. Such enriched communication may form the basis of multi-agent 
collaborative negotiation (Chu-CarroU & Carberry, 1996). Currently, STEAM relies on the 
team or subteam leader when resolving disagreements, particularly when deciding the next 
action. While leadership in teamwork is by itself an interesting phenomena of investigation, 
enabling agents to negotiate their plans without a leader would also improve STEAM's 
fiexibility. We hope that addressing such issues would ultimately lead STEAM towards 
improved fiexibility in teamwork. 
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Appendix A: Detailed STEAM Specification 

The pseudo-code described below follows the description of STEAM provided in this article. 
It is based on execution of hierarchical operators, or reactive plans. All operators in the 
hierarchy execute in parallel, and hence the "in parallel" construct. The comments in the 
pseudo code are enclosed in /* */. The terminology is first described below, to clarify the 
pseudo-code. 

• Execute- Team- Operator(a, 0, C, {pi, p2,...,pn}) denotes the execution of a team 
operator a, by a team 0, given the context of the current intention hierarchy C, and 
with parameters pi, p2...pn. 

• Terms 7, Cme, Cc, r, Cmt are all exactly as in Section 5. 

• [aje denotes the team 0's joint intention to execute a. 

• status([Q!]e, STATUS-OF-a) denotes the status of the joint intention [aje, whether 
it is mutually believed to be achieved, unachievable or irrelevant. 

• satisfies (Achievement-conditions(Q!), f) denotes that the fact / satisfies the achieve- 
ment conditions of the team operator a; similarly with respect to unachievability and 
irrelevancy conditions. 

• CommMn«cate(terminate-jpg(Q!), /,0) denotes communication to the team to termi- 
nate 0's joint commitment to a, due to the fact f. 

• Update-state (team-state(0), f) denotes the updating of the team state of with 
the fact f. 

• Update-status([Q!]e) denotes the updating of the team operator a with its current 
status of achievement, unachievability or irrelevancy. 

• Agent(Q!) is the individual agent or team executing operator a. 

• actions(Q!) denote the actions of the operator a. 

• teamtype(t/') is a test of whether the agent tp is a team or just one individual. 

• self(t/') is a test of whether the agent tp denotes self. 

• agent-status-change(;u) denotes change in the role performance capability of agent 
or subteam p. 

• Execute-individual- Operator(tJj, self, C, {pi, p2,...,pn}) denotes the execution of an 
individual operator tp by self, given the context of the current intention hierarchy C, 
and with parameters pi, p2...pn. 

For expository purposes, "Execute-team-operator" and "Execute-individual-operator" 
are defined as separate procedures. In reality, STEAM does not differentiate between the 
two. 
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Team Operator Execution 

Execute- Team- Operaior(a, 0, C, {pi, p2,...,pii}) 
{ 

1. estimate 7; /* See Section 5. */ 

2. if 7* Cme > Cc execute establish commitments protocol; 
/* see Section 4.1 for explanation. */ 

3. establish joint-intention [a]©; 

4. While NOT(status([a]0, Achieved) Y status([a]0, Unachievable) Y status([a]0, Irrel- 
evant)) Do 

{ 

(a) if (satisfies (Achievement-conditions(a), f) Y satisfies (Unachievability-conditions(a), 
f) Y satisfies (Irrelevance-conditions(a), f)) 

/* This is the case where fact f is found to satisfy the termination condition of a. The 
case where f is only a threat to a (see Section 5.1) is analogous. */ 

{ 

i. estimate r; /* see section 5. */ 

ii. if r* Cmt > Cc propose-operator CommMM«ca<e(terminate-jpg(a), /, 0) with high 
priority; 

/* See Section 5 and 4.1*/ 

iii. if no other higher priority operator, in parallel 
Execute-individual-operator(Communicate(teiinmate-i];>g(a), f, 0), self, a/C, {pi, 

p2,...}y, 

iv. Update-state (team-state(0), f); 
V. Update-status([a]0); 

} 

(b) if agent-status-change(//), where G 
{ 

i. Evaluate role-monitoring constraints; /* See Section 4.2. */ 

ii. if role-monitoring constraint failure cf such that (satisfies (Unachievability-conditions( 
cf) then update-status([a]0); 

} 

(c) if receive communication of terminate-jpg(a) and fact / 
{ _ 

if (satisfies (Achievement-conditions(a), f) Y satisfies (Unachievability-conditions(a), 
f) Y satisfies (Irrelevance-conditions(a), f)) 

{ 

i. Update-state (team-state(0), f); 

ii. Update-status([a]0); 

} 
} 

(d) Update-state(team-state(0), actions(a)); 

/* execute domain-specific actions to modify team state of */ 
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(e) if children operator /31,/32,.../3n of a proposed as candidates 
{ 

i. /3i ^ select-best{/31.../3n}; 

ii. if (teamtype(Agent(/3i)) f\ (Q = Agent(/3i))) then in parallel 
Execuie-ieam-operaior(pi, 0, a/C, {pl,p2...}); 

iii. if (teamtype(Agent(/3i)) /\ (Agent(/3i))c 0) then in parallel 
{ 

A. Execuie-ieam-operaior(pi, Agent(/3i), a/C, {pl,p2...}); 

B. Instantiate role-monitoring constraints; 

} 

iv. if self(Agent(/3i)) then in parallel 
{ 

A. Execuie-mdividual-operaior(pi, self, a/C, pi...); 

B. Instantiate role-monitoring constraints; 

} 

} 

} /* End while statement in 4 */ 

5. terminate joint intention [a]©; 

6. if status([a]0, Unachievable) 

{ 

if (a != Repair) /* If a is not itself Repair */ 
{ 

Execuie-ieam-operaior( Repair, Q, C, {a, cause-of-unachievability,...}) 

/* Repair is explained in detail in Section 4.2. Cause-of-unachievability, passed as a parameter 
to Repair, may be role-monitoring constraint violation as in case 4b, or the domain-specific 
unachievability conditions. */ 
} else { 

E X ecut e-t earn- operator{ Complete- Failure, Q, C, {a, cause-of-unachievability,...}) 
/* If Repair is itself unachievable, complete-failure results, as in Section 4.2 */ 

} 
} 

} /* end procedure execute-team-operator */ 

Individual Operator Execution 

Execute-individual- Operator{'il> , self, C, {pi, p2,...,pn}) 
{ 

1. establish tp as an individual intention; 

2. While NOT(status(i/', Achieved) Y status(i/'. Unachievable) Y status(i/'. Irrelevant)) 

Do 
{ 

(a) if (satisfies (Achievement-conditions(i/'), f) Y satisfies (Unachievability-conditions(i/'), 
f) Y satisfies (Irrelevance-conditions(i/'), f)) 
{ 

i. Update-state (state(self), f); 
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ii. Update-status(i/'); 

} 

(b) Update-state(state(self), actions(i/')); 

/* execute domain-specific actions to modify private state */ 

(c) if new children operator {/31.../3n} oi tp proposed 
{ 

i. j3\ ^ select-best{/31.../3n}; 

ii. Execuie-mdividual-operaior(pi, self, tp/C, {pi...}) 

} 

} /* end while statement in 2 */ 

3. if status(i/', Unachievable) 

{ 

if (ip != Repair) 
{ 

Execuie-mdividual-operaior( Repair, self, C, {ip, cause-of-unachievability,...}) 

/* Repair is explained in detail in Section 4.2. Cause-of-unachievability is only domain-specific 

unachievability condition. This is passed as a parameter to repair. */ 

} else { 

Execuie-individual-operaior(Compleie-Failure, self, C, {ip, cause-of-unachievability,...}) 

/* If Repair is itself unachievable, complete-failure results, as in Section 4.2 */ 

} 

} 

} /* end procedure execute-individual-operator */ 
Appendix B: STEAM Sample Rules 

The sample rules described below follow the description of STEAM provided in this article, 
and essentially help encode the algorithm described in Appendix A. The rules, as with the 
algorithm in Appendix A, are based on execution of hierarchical operators, or reactive plans. 
While the sample rules below are described in simplified if-then form, the actual rules are 
encoded in Soar, and are available as an online Appendix. 

SAMPLE:RULE:CREATE-COMMUNICATIVE-GOAL-ON-ACHIEVED 

/* This rule focuses on generating a communicative goal 
if an agent's private state contains a belief that satisfies 
the achievement condition of a team operator [OP]©. 
See section 4.1. */ 
IF 

agent i^i's private state contains a fact F 
AND 

fact F matches an achievement condition AC 

of a team operator [OP]© 

AND 

fact F is not currently mutually believed 
AND 

a communicative goal for F is not already generated 
THEN 
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create possible communicative goal CG to communicate fact F to team 
to terminate [OP]©. 

SAMPLE:RULE:CREATE-COMMUNICATIVE-GOAL-ON-UNACHIEVABLE 

/* This rule is similar to the one above. */ 
IF 

agent i^i's private state contains a fact F 
AND 

fact F matches an unachievability condition UC 

of a team operator [OP]© 

AND 

fact F is not currently mutually believed 
AND 

a communicative goal for F is not already generated 
THEN 

create possible communicative goal CG to communicate fact F to team 
to terminate [OP]©. 

SAMPLE:RULE:ESTIMATE-VALUE-FOR-NON-COMMUNICATION 

/* This rule estimates T*Cmt for non-communication 
given a communicative goal, using the formula from 
Section 5.*/ 
IF 

CG is a possible communicative goal to communicate fact F to team 

to terminate [OP]© 

AND 

Cmt is estimated high 
AND 

r is estimated low 
THEN 

Estimated value of non-communication is medium. 
SAMPLE:RULE:DECISION-ON-COMMUNICATION 

/* This rule makes the communication decision using the formula * with T*Cmt and Cc 
from Section 5.*/ 

IF 

CG is a possible communicative goal to communicate fact F to team 

to terminate [OP]© 

AND 

Estimated value of non-communication for CG is medium 
AND 

Estimated value of communication for CG is low 
THEN 

post CG as a communicative goal to communicate fact F to team 
to terminate [OP]© 

SAMPLE:RULE:MONITOR-UNACHIEVABILITY:AND-COMBINATION 

/* This rule checks for unachievability of role-monitoring 
constraints involving an AND-combination. See section 4.2. 
/ 
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IF 

A current joint intention [OP]© involves an AND-combination 
AND 

i>i is a member performing role to execute sub-operator op 
AND 

no other member i>i is also performing role to execute sub-operator op 
AND 

i>i cannot perform role 
THEN 

Current joint intention [OP]© is unachievable, due to a critical role failure 
of i>i in performing op 
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